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ABSTRACT 

Defining metacognition as the set of skills and 
strategies one uses in monitoring and modifying how one learns, this 
paper addresses ways in which interactive hypermedia/multimedia 
instructional programs might enhance the meta-cognitive abilities of 
the learners who use them. It begins by identifying key issues and 
approaches to metacognition, including the internal and external 
structures of hypermedia/multimedia products. External structure is 
described as the ways in which the product assists the learner in 
gaining access to and making use of the nodes in the product (the 
internal structure) , and strategies for supplying external structure 
are suggested. Devices for measuring the extent to which learners' 
metacognit ive activities are stimulated, exercised, and developed are 
discussed, including journal entries, progress logs, transaction 
shell data, oral discussion, and learner created materials. Specific 
metacognitive skills are then examined under six broad headings: Task 
Analysis, Goal Setting, Strategic Action, Cognitive Load, Persistence 
and Responsibility, and Metacognitive Growth. Within each heading, 
specific metacognitive skills that might enhance learners 1 use of 
hypermedia/multimedia instructional products are examined. For each 
skill, the paper examines two approaches that hypermedia/multimedia 
products have used or might use to encourage the development of that 
skill. The paper next addresses what types of data one might expect 
to find as evidence uf the operation of that skill. Along the way, 
issues of import in identifying, testing, and evaluating learner 
metacognition are discussed. (Contains 118 references.) (BBM) 
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Part I: Introduction 

Goetz (1984) contended that "within the past 20 - 30 years, 
the cognitive-information-processing view of learning has 
emerged and . . . learning is seen as the product of com- 
plex, interrelated cognitive operations which greatly trans- 
form the information being processed" (p. 50). Writers and 
researchers who adhere to the in formation -processing model 
of learning talk about cognitive load and the burden that it 
can impose on learners (Phye & Andre, 1986). Cognitive 
load is based largely on the limitations of short-term or 
working memory (Fleming & Levie, 1978; Miller, 1956). 
Access to increasingly larger and more frequently revised 
data sets (clusters of related information) makes cognitive 
overload increasingly more probable and emphasizes the 
need for efficient and efficacious strategies for examining 
this material (Charp, 1986; White, 1988). 

At the same time, one frequently finds developers of 
hypermedia and multimedia instructional programs arguing 
for learner "empowerment" and learner control (Florin, 
1990; Hannafin & Richer, 1989b; Lee, 1990). While such 
freedom and control may offer learners opportunities to 
make novel connections and Icam in divergent ways 
(Jonassen, 1991; Marchionini, 1988), it also offers oppor- 
tunities for them to experience substantial cognitive overload 
and become disoriented with no sense of location, direction, 
or purpose (Case, 1980a; Gygi, 1990; Heller, 1990; Morariu! 
1988; Oren, Salomon, Kreitman, & Don, 1990). 

One scarcely need note that the thinking skills of learners 
are the focus of frequent, and frequently negative, articles. 
Learners (and either indirectly or directly, teachers and the 
educational system) are assailed regularly for learners* 
reported inabilities to form and test hypotheses, draw 
conclusions, and evaluate in a critical manner the materials 
to which they are exposed (Dee & Barkley, 1989; DeBower 
& DeBower, 1990; Flesch, 1991; Haywood, 1987; Marzano, 
1987; Robert, Racine, & Bowers, 1990; Stinespring, 1991). 
Such concerns about our educational system and its learners 
are not new, however (see for example, Flexner, 1917; 
Franklin, 1779; Judd, 1933; Monroe, 1907). As evidence 
of this longstanding concern, or as a consequence of it, 
many writers and researchers have devoted their energies to 
proposing, identifying, or testing new approaches to helping 
learners enhance their thinking skills. This body of work 
offers much of value to those considering thinking skills* 



relation to instructional hypermedia/multimedia products, 
particularly in relation to Goetz's earlier-mentioned interre- 
lated cognitive processes that transform raw data into 
meaningful and useful information. 

Some writers argue that we need either to develop a new set 
of skills specifically designed to help learners handle the 
cognitive demands of the "information age," or to refine and 
refit current skills to meet those demands (see for example, 
Card, Moran, & Newell, 1983; Case, 1980b; Cates, 1990; 
Cates, 1991b; Hannafin & Rieber, 1989a). The global term 
used to refer to the set of skills and strategies one uses in 
monitoring and modifying how one learns is metacognition. 
Perhaps the most frequently cited definition of metacogni- 
tion is that of Flavell (1976): 

Metacognition refers to one's knowledge 
concerning one's own cognitive processes 
and products or anything related to them, 
e.g., the learning-relevant properties of 
information or data. . . . Metacognition 
refers, among other things, to the active 
monitoring and consequent regulation and 
orchestration of these processes in relation 
to the cognitive objects or data on which 
they bear, usually in the service of some 
concrete goal or objective, (p. 232) 

It appears logical that the skills that might enable learners 
to make best use of the resources offered by hypermedia/- 
multimedia products are those skills that fall under the 
heading of metacognition. How then, should these instruc- 
tional products stimulate, exercise, and develop learner 
metacognition? 

This paper addresses ways in which interactive hyper- 
media/multimedia instructional programs might enhance the 
metacognitive abilities of the learners who use them. It 
approaches the topic from an evaluative researcher's point 
of view. That is, this paper seeks first to identify key issues 
and approaches to metacognition and then examines meta- 
cognitive skills under six broad headings: Task Analysis, 
Goal Setting, Strategic Action, Load, Persistence and 
Responsibility, and Growth. Within each heading, specific 
metacognitive skills that might enhance learners' use of 
interactive hypermedia/multimedia instructional products are 
examined. For each skill, the paper examines two ap- 



ERLC 



Paper presented at the annual meeting of the American Educational Research Association, April 20, 1992, San Francisco, CA. 

2 BEST COPY AVAILABLE 



Metacognition in Interactive Hypermedia/Multimedia Instructional Products I Page 



proachcs which hypermedia/multimedia products have used 
or might use to encourage the development of that skill. 
The paper next addresses what types of data one might 
expect to find as evidence of the operation of that skill. 
Along the way, issues of import in identifying, testing, and 
evaluating learner metacognition will be discussed. 

Part II: Structure and Metacognition 

Structure in a hypermedia/multimedia product may be of 
two types: internal and external. Internal structure refers 
to the way in which the nodes (individual pieces of infor- 
mation or clusters of information) within the product arc 
organized {linked). External structure refers to the ways in 
which the product assists the learner in gaining access to 
and making use of the nodes in the product. Using an 
arbitrary dichotomy, one may divide each type of structure 
into two levels: high and low. 

Products having low internal structure would be composed 
of loose affiliations of nodes with few explicit links, or with 
links whose logical connections were unclear. Many single- 
link nodes or omni-link nodes might exist in such a product 
A single-link node is connected to only one other node by 
a single link that may or may not be bidirectional. Omm- 
link nodes are connected to all other nodes in the product 
either by bidirectional links, or by "near links" that take the 
learner no more than one or two nodes from the original 
linked node before offering a return path (link). 

Products having high internal structure would employ highly 
organized node structures, often in either hierarchical or 
network form. Relationships of dependency, superordinance 
and subordinance, and ancillary and auxillarily might well 
typify the links among nodes (Entwistle, 1981). In the more 
highly ordered products of this type, multi-linked nodes 
would be the rule, rather than the exception, and one might 
expect to find few nodes with single links. One would not 
expect, however, to find many omni-linked nodes, since it 
is difficult to create highly structured networks in which 
such near omni-directionality can occur. 

Products having low levels of external structure would 
incorporate few explicit representations of internal structure, 
few, if any, advisement or guidance features, and little 
navigational assistance. In contrast, products having high 
levels of external structure might well include explicit, 
representations of the internal structure of the product, many 
advisement and guidance features, and much navigational 
aid. 

Using this taxonomy, one could logically divide products 
into four possible structural permutations: (1) low internal 
and low external structure; (2) high internal and low 



external structure; (3) low internal and high external 
structure; and (4) high internal and high external structure. 
Operational concerns and properties of each of these types 
of products are discussed below. 

I ow Internal Structure - Low Exte rnal Structure 

Learners would most likely have difficulty representing the 
contents of such products using logically-ordered arrange- 
ments, such as flowcharts, hierarchies, or networks, since 
the developers had not explicitly designed the products to 
meet that level of internal structure. One might well find it 
possible to create representational "fragments," however. 
Such fragments would represent those nodes for which some 
logical relationship could be determined or for which some 
relationship could be inferred. Additionally, one would not 
expect such products to supply support mechanisms to assist 
learners in deciphering either the structure of the product or 
relationships among nodes, since these types of internal 
structure are lacking or imperfectly understood. Florin 
(1990) calls such products data collections. They may be 
intended for use as resource materials, but their low levels 
of structure may impose substantial metacognitive burdens 
on their learners. These products are often characterized by 
fruitless searches, learner disorientations, and a tendency to 
produce disjointed outcomes (Jonassen & Grabinger, 1990; 
Marchionini, 1988; White, Cates, & Fontana, 1991). 

Hip ;h Internal - Low External Structure 

One might expect these products to be logically organized, 
with linkages that are both logical and multi-directional. If 
one could ascertain the underlying internal structure,^ it 
should be possible to create representations (schemata) that 
illustrate the nature and organization of both nodes and 
links. Logical order may well act in opposition to density 
or complexity in these products, however. (Density refers to 
the number of nodes in a product, while complexity refers 
to the number and intcrdependency of links among nodes in 
the product.) While the logical arrangement of the pro- 
duct's nodes may act to make the system "predictable," if 
the level of density or complexity becomes too great, 
learners may be overloaded by the quantity of nodes, 
interdependency of links, or both. In short, a dense or 
complex network may leave learners unable to grasp the 
underlying logic and may invite confusion of the "now what 
was I doing" sort (Gagne & Briggs, 1979; Gygi. 1990; 
Lanza & Roselli, 1991; Oren et al, 1990). The low level of 
external structure leaves learners dependent upon their own 
perceptions of the moire, or pattern, of the nodes and links 
(Heller, 1990). One would expect denser or more complex 
products in this group to place higher metacognitiv de- 
mands on learners than less dense or less complex products. 
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Low Internal - High External Structure 

Low levels of internal structure may make it difficult for 
learners to identify the underlying structure and this, in turn, 
may make it difficult to ascertain the logic of linkages 
among nodes. High levels of external structure should 
provide substantial assistance in using the product, however, 
provided that the external support supplied is both appropri- 
ate and timely. While a high level of external structure 
should reduce the metacognitive load of these products as 
compared with comparable products exhibiting low levels of 
both types of structures, the absence of internal structure 
may not be easily overcome. That is, learners' inability to 
perceive internal structure may result in their becoming 
dependent upon the external structure for guidance and 
advice. If this proves to be the case, such dependency acts 
in opposition to metacognition, for metacognitive skill is, by 
definition, self-sufficient (Gavaiek & Raphael, 1985). 

High Internal Structure - High External Structure 

Products in this group appear to be those best suited to 
metacognitive development, exercise, and growth. They 
exhibit both die high level of internal structure that makes 
it possible for learners to identify on their own the content 
organization and existing relationships and the high level of 
external structure that makes that organization and those 
relationships apparent to the learner. In addition, guidance 
and advisement features, if properly employed, should 
facilitate learner metacognitive activity. 

The discussions that follow address devices and strategies 
related to both internal and external structure. While 
instructional hypermedia/multimedia products that employ 
both high internal structure and high external structure may 
appear to be the best canoidates for metacognitive stimula- 
tion, the discussions that follow should be more or less 
applicable to products falling into the three other structural 
permutations as well. The absence of evidence confirming 
the operation of metacognition, or the present of evidence 
suggesting metacognitive failure, may tell the researcher 
much about a product. Perhaps such findings might be 
interpreted in relation to the combinations of internal and 
external structure described above. Additionally, the final 
part of this paper will discuss metacognition and learner 
dependency on external structure. 

Part III: Supplying External Structure: 
Support Strategies 

In a hypermedia/multimedia product, external structure may 
be supplied to the learner in a variety of ways. For purpos- 
es of this paper, these ways will be classified under two 



categories: Static/Directive Support and Dynamic/Inter- 
active Support. The two category names are intended to be 
descriptive of the underlying philosophies and approaches 
employed by the two types of support. The names were 
selected on the basis of their definitions, Static refers to 
that shows little change and tends to remain in the same 
location. Directive refers to something that serves to direct, 
guide, govern, or influence (Webster's, 1976), Thus, static 
or directive support devices would be those evidencing the 
following characteristics: 

1. They are almost always displayed on screen or are 
almost always available, 

2. They are generally under program control rather 
than learner control (Belland, Taylor, Canelos, 
Dwyer, & Baker, 1985; Gray, 1987; Ho, Savenye, 
& Haas, 1986; Kinzie, Sullivan, & Berdcl, 1988; 
McNeil & Nelson, 1991; Yore, 1986), 

3. They are usually generic rather than specific in 
nature. 

4. They favor direct instruction over indirect instruc- 
tion or modeling (Parker, 1991). 

5. They arc usually displayed in the same form each 
time that they appear, regardless of the identity of 
the learner or the number of times they have 
appeared previously to the same learner. 

Dynamic devices are marked by continuous and usually 
productive activity or change and are noted for being both 
energetic and forceful. Interactive refers to something 
characterized by mutual or reciprocal action or influence 
(Webster's, 1976). Thus, dynamic or interactive support 
devices would be ones evidencing the following character- 
istics: 

1. They are usually context-sensitive and appear only 
when appropriate (Hutchings, Hall, Briggs, Ham- 
mond, Kibby, McKnight, & Riley, 1992). 

2. They are generally under learner control rather than 
program control (Hannafin & Rieber, 1989b; 
Kinzie & Sullivan, 1989; Lee, 1990; Milhcim & 
Martin, 1991; Whitener, 1989), 

3. They usually offer specific advice n .her than 
generic advice (Lee, 1991; Wilson & Cole, 1991). 

4. They favor modeling and illustration (indirect 
instruction) over direct instruction (Brown, 1987; 
Parker, 1991). 

5. They may be adaptive across repeated use by a 
single learner or across different learners (Keller, 
1987; Lehrer & Randle, 1987; Ross & Morrison, 
1989). 
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Part IV: Measurement Devices 



This part addresses only the question of how to measure the 
extent to which learners' metacognitive activities are 
stimulated, exercised., and developed in instructional 
hypermedia/multimedia products. It does not address the 
thorny issue of whether metacognitive activity, in and of 
itself, produces higher levels of learning of content material. 
For purposes of this paper, we will assume that learners so 
stimulated will complete the materials at an acceptable level 
of performance in ways that meet the objectives or intents 
set for them, and that they will do so within an acceptable 
range of completion times. While these are indeed grand 
assumptions, they are made here solely for the purpose of 
isolating the issue of identification and measurement from 
the broader issue of effectiveness. We recognize that 
effectiveness is a crucial issue. One has difficulty conceiv- 
ing, however, how one can determine effectiveness until 
data that might be used as evidence of the operation of 
specific metacognitive skills have been identified. Thus, the 
present paper is forced either to tackle untying the Gordian 
Knot or, as Alexander the Great did, to cut it as a first act 
(Bulfinch, 1968). If the present work proves useful to 
others investigating the efficacy question, it will have served 
its intended purpose (Cates, R.S., 1990). 

For purposes of this paper, measurement will be considered 
in relation to two categories of evidence: process evidence 
and product evidence. Process evidence is evidence 
gathered during the process of learning. Since such evi- 
dence often does not produce tangible products, it may 
normally be overlooked. Pressley,Borkowski,&O f Sullivan 
(1985) a*id Slee (1989) argued, however, that this evidence, 
may be of great value in evaluating learner metacognition. 
Product evidence consists of materials and products that are 
produced as a result of working through the learning 
process. Of the measurement devices discussed below, oral 
discussion is the only one that is used exclusively in 
gathering process data. The other four devices may be used 
to gather both process and product data. 

Journal Entries 

Many hypermedia/multimedia products incorporate an 
electronic journal function. This journal is actually a small 
word processor in which learners can write notes to them- 
selves, retain for their own use copies of materials in the 
product (such as text files or graphic images), or print out 
copies of materials for use outside the product. If a product 
does not provide such an electronic journal, one might still 
use a traditional paper-and-pencil journal to record one's 
thoughts. There is a long tradition of the use of journals in 
a variety of subject areas, particularly the humanities, and 



the inclusion or use of journals is often justified on the 
grounds that it encourages "reflection," "insight," or meta- 
cognitive awareness (Bransford & Vye, 1989; Collins, 
Brown, & Newman, 1989; Glatthorn & Baron, 1985; Jones, 
1992). Journal entries or notations, whether they be 
recorded electronically or mechanically, seem to offer a 
legitimate source of evidence of metacognitive activity. 

Analyzing journal entries as evidence of metacognition is 
not without problems, however. First, researchers analyzing 
such data will need to establish clearly which types of 
entries and notations are indicative of which types of 
metacognitive activity. This introduces a large measure of 
subjectivity in analysis, but subjectivity is not necessarily 
problematic, provided that researchers make clear the bases 
for their interpretations (Bogdan & Biklen, 1982; Borg & 
Gall, 1989). Second, learners may not be conscientious in 
writing in their journals. If they are not, much data that, 
might illustrate metacognitive activity may go unrecorded. 
Third, learners may lack candor in their entries, particularly 
if they know or think that their entries may later be the 
subject of another's scrutiny. Fourth, learners may lack the 
maturity, insight, or writing skills to express what it is that 
they are thinking or feeling. If this proves true, their journal 
entries may offer an impoverished resource for researchers. 
This impoverishment may be further complicated if journals 
are maintained electronically, since for some students the 
impediment of keyboarding or using the features of a word 
processor may be inhibiting. The added mechanical load 
may make it difficult for these students to concentrate on 
their own mental operations when almost all of their spare 
mental capacity is consumed by the mechanical task of 
making journal entries (Phye & Andre, 1986). 

Progress Logs 

Many programs are capable of maintaining a record of 
where learners are at present and where they have been. 
Often hypermedia/multimedia programs maintain this record 
in the form of "recent" maps. Such data may be recorded 
on disk as progress logs for later analysis. These logs may 
allow researchers to reconstruct the paths learners take 
through a program and can help researchers infer the 
metacognitive activity involved. 

Transaction Shell Data 

Similarly, there are programs called transaction shells which 
operate in the background while a learner is working with 
a program. Transaction shells are not only capable of 
recording where a learner has been, but also what he or she 
did while there. These "transactions" can then be examined 
with an eye to inferring or deducing which mental activities 
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(and more specifically, mctacognitive activities) appear to 
account for what the transaction shell data show. 

Oral Discussion 

A fourth source of data on metacognitive activity may be 
oral comments made by the learner, either in conversation 
or in isolation. Day, French, and Hall (1985) argued that 
metacognitive skills are refined in social interaction. This 
contention finds much support in the professional literature 
(Baccker & Small, 1990; Baron & Kallick, 1985; Bransford 
& Vyc, 1989; Brown, 1987; Costa, 1985a; Gavalek & 
Raphael, 1985; Moss, 1990; Prawat, 1989). One source of 
such refinement may be the use of cooperative groups which 
interact with one another while working on computer 
(Dalton, Hannafin, & Hooper, 1989; Del Marie Rysavy & 
Sales, 1991; Gomoll, 1990) Photographic, videotaped, 
audio-recording, or simply overheard versions of these oral 
transactions offer rich sources of research data (Baron & 
Kallick, 1985; Tucker & Dempsey, 1991). 

Learner-Created Materials 

Learners may create a variety of materials for their own use 
as they work through hypermedia/multimedia products. 
These materials may also present rich sources of data for 
researchers. 

Part V: Specific Metacognitive Skills: Sup- 
port Strategies and Possible Evidence of 
Metacognitive Activity 

This part of the paper addresses eleven specific meta- 
cognitive skills falling into six broad categories. A 
separate section will be devoted to each of the six broad 
categories. Each section will utilize a similar structure of 
subsections. The section will begin by identifying a 
metacognitive skill. The next two subsections will iden- 
tify specific ways in which an instructional hypermedia/- 
multimedia product might address this skill, first using 
static/directive support and then dynamic/interactive 
support. The next two subsections discuss anticipated 
evidence of the operation of the metacognitive skill: 
process evidence and product evidence. The final sub- 
section for each skill discusses any concerns related to 
skill. If a section contains more than one metacognitive 
skill, each skill is addressed in the manner described, in 
turn, before going on to skills in the next one of the six 
broad categories. Where appropriate, relevant references 
are cited. 



Task Analysis 

Skill : Recognizing the size and scope of a 

task (Greeno & Riley, 1987; Lawson, 1980). 

Static/Directive Support : 

The key issue here is helping the learner to get an ade- 
quate image of the internal structure of the product, while 
at the same time focusing the learner's attention on the 
specific task at hand (Psotka, 1991). Support devices that 
might serve this purpose include various forms of advance 
organizers. Promising advance organizers include: 

1. Outlines of the steps or stages in the task (Krahn 
& Blanchaer, 1986); 

2. Content maps that attempt to make clear the 
interrelationships of the content to be examined 
(Collins et al, 1989; Heller, 1990; Marchionini, 
1989); 

3. Learning objectives that specify exactly what the 
learner is to accomplish and how accomplishment 
is to be demonstrated (Hannafin & Rieber, 
1989b; Ho et al, 1986); 

4. Supplied schedules or time estimates designed to 
help the learner project how long it will take to 
complete the task (Cates, 1991c); 

5. Adjunct questions that call for learners to for- 
mulate schemata (Gordon & Braun, 1985; 
Schloss, Sindeiar, Cartwright, & Schloss, 1986). 

Dynamic/Interactive Support : 

The key here is helping learners monitor their progress 
toward completion of the task. One promising approach 
would be for the program to monitor the learner's prog- 
ress and the amount of time expended. The program 
could then compare these data to averages of time and 
progress and offer advice to the learner on the basis of 
these comparisons. The intent here is to alert the learner 
to the rate of progress and to offer advice that clarifies the 
scope of the task. Perhaps such advice should be mod- 
eled on the performance of three types of learners: begin- 
ner, intermediate, and expert The advice could then be 
based on the match between the present learner's progress 
and that of the models (Carrier & Jonassen, 1988; Hanna- 
fin & Rieber, 1989b; Keller, 1987; Psotka, 1991). 

Process Evidence : 

Transactions: Evidence that the learner is exercising this 
metacognitive skill in the process of completing the task 
would be records of transactions involving advance orga- 
nizers which were available. Repeated access to such 
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advance organizers, particularly those related to schedul- 
ing and time estimates might be particularly diagnostic, as 
might access to such devices anytime the learner resumes 
work on the task following a break. 
Oral: Process evidence here might consist of discussions 
or comments about the nature and scope of the task. As 
is the case with all oral data, such discussions or com- 
ments might occur in teacher-student interactions, in 
cooperative student task groups, in isolation, or in inci- 
dental conversation. 

Product Evidence: 

Journal: Journal notes or entries referring to the nature, 
scope, or size of the task suggest the operation of this 
skill. 

Learner-Created: Learners might draw representations of 
the task or attempt to create their own reconccptualiza- 
tions of the task at hand. Such representations, whether 
formally drawn or merely sketched out, suggest the opera- 
tion of this mctacognitive skill (Goetz, 1984; Hannafin & 
Riebcr, 1989a; Kearsley, 1988; Young, 1983). 

Concerns: 

An obvious concern here is that time and progress com- 
parison values will be difficult to derive and, regardless of 
the accuracy of the comparison values, metacognition is 
not a comparison-based activity. In other words, consider 
for a moment the expression "His mill grinds slow, but it 
grinds exceedingly fine." Speed is not automatically a 
measure of excellence, nor should it be assumed to be a 
measure of metacognition. Learner models will need to 
be well-designed and highly generalizable if they arc to 
be valid and useful. 

In order to make the most of the devices and advice 
offered, learners need at least minimal awareness of their 
own task behaviors (work habits, rate of progress). Oth- 
erwise, they may not recognize which advice to take and 
which to ignore (Catcs, 1991b). Similarly, learners may 
need training in advance in order to know how to draw 
representations and reconceptualizations (Card et al, 
1983). 

Goal Setting 

Skill: Setting appropriate goals and subgoals 

(Goetz, 1984; Greeno & Riley, 1987; Prawat, 
1989). 



Static/Directive Support: 

The program could suggest formal goals and subgoals 
(Baird, 1988). It might use direct instruction to point out 
how goals and subgoals contribute to completion of the 
task. 

Dynamic/Interactive Support : 

The program could offer the learner a selection of possi- 
ble goals and related subgoals, allowing the learner to 
select among them as desired. The program would then 
retain a record of the goals and subgoals selected and 
could tailor the advice it offers to the selected goals and 
subgoals (Keller, 1987; Zellermayer, Salomon, Globcrson, 
&Givon, 1991). 

Process Evidence : 

Transactions: Indications of the operation of this meta- 
cognitive skill might include access or repeated access to 
formally presented goals or subgoals, selection of goals 
and subgoals from an offered list, or access to offered 
advice on goals and subgoals. 

Oral: Comments or discussions relating to goals or sub- 
goals suggest metacognitive skill activation. 

Product Evidence : 

Journal: Entries or notations would address goals and 
subgoals. Such entries might merely list them, or might 
comment on their formation. 

Learner-Created: Hierarchical layouts or network illus- 
trations of the relationships of goals and subgoals suggest 
initial metacognition in goal setting. 

Concerns : 

Before we can expect learners to participate in setting 
goals, they must understand what goals and subgoals are, 
and how they relate to one another in leading to the ac- 
complishment of a task. Learners may also need to be 
taught how to represent goals and subgoals in hierarchies 
and networks as part of their understanding of how they 
are related (Glynn & Di Vesta, 1977). 

Skill: Revising goals and subgoals as neces- 
sary (Gavalek & Raphael, 1985; Glatthorn & 
Baron, 1985; Wellman, 1985). 
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St a t ic/Directive Support : 

The program might provide learners with the ability to 
view goals and subgoals as they work toward completion 
of the task (Wolz, McKeown, & Kaiser, 1989). This 
might be accomplished by a goals/subgoals icon that 
permitted the learner to review goals and subgoals as 
desired (Cates, 1991c). Through an intervening coach, the 
program could impose external evaluation of goals and 
subgoals with mandated revisions (O'Shea & Self, 1983; 
Marchionini, 1989; Poppen & Poppen, 1988). 

Dynamic/Interactive Support: 

The program could monitor learner progress and suggest 
revisions at appropriate points. The locations of these 
revisions points, the bases for monitoring learner progress, 
and the nature of the advice to be offered could once 
again be based upon comparison to the three learner 
models discussed earlier (Yordy, 1991). The program 
could also pose a series of questions designed to help the 
learner assess the adequacy of the goals and subgoals 
currently selected (Bellanca, 1985; Costa, 1985a; Day et 
al, 1985). 

Process Evidence : 

We would expect to see repeated access to goal and 
subgoal statements and to hear oral comments and discus- 
sions questioning the adequacy and appropriateness of 
those goals and subgoals. 

Product Evidence : 

Journal: Entries and notations would address the ade- 
quacy of the goals and subgoals selected and might sug- 
gest revisions. 

Transactions: The learner would have resclected goals or 
subgoals or substituted a new set of goals and subgoals. 

Concerns : 

Learners need experiences in judging the adequacy and 
appropriateness of goals and subgoals. This will require, 
many instances of exercise followed by debriefings. 
While this may occur in using hypermedia/multimedia 
products, it is unlikely that the computer program can do 
an adequate job of debriefing, and thus teacher interven- 
tion will be required (Cates, 1991b). 



Strategic Action 

Skill ;- Selecting appropriate learning strate- 
gies (Deny, 1985; Deny, 1989; Lawson, 1980; 
Pressley et al, 1985). 



Static/Directive Support : 

The program could prescribe strategy. It could supply 
direct instruction in how to select and use strategies 
(Beyer, 1991; Parker, 1991). It could impose strategy 
through an intervening coach (Bransford & Vye, 1989; 
O'Shea & Self, 1983). 

Dynamic/Interactive Suppo rt: 

The program could offer strategic advice (Gavalek & 
Raphael, 1985). It could offer learners opportunities to 
view modeling of strategic actions (White, 1989). The 
program could pose questions designed to help focus the 
learner's attention on the key selection issues (Collins et 
al, 1989). 

Process Evidence : 

Transactions: There would be evidence of learners hav- 
ing gained access to offered presentations on strategy or 
advice on strategic actions. 

Oral: Learners would comment on or discuss possible 
learning strategies. 

Product Evidence : 

Journal: Entries or notations on learning strategies would 
appear. 

Concerns : 

Learners may have difficulty grasping what each strategy 
does and what makes a strategy appropriate in one context 
and not in another. 

Skill : Determining the effectiveness of a 
learning strategy or set of strategies 
(Deny, 1985; Deny, 1989; Goetz, 1984; Lawson, 
1980; Presseisen, 1985; Wellman, 1985). 

Static/Directive Support : 

The program could provide generic introductions on the 
use of strategies. It could provide direct instruction on 
evaluating strategies (Beyer, 1988). 
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DynamicAnteractive Support : 

Once again, the program could offer strategic advice or 
offer learners opportunities to view modeling of strategic 
evaluations (Costa, 1985a). The program could pose 
questions designed to help focus the learner's attention on 
key effectiveness issues (Bellanca, 1985; Day etal, 1985; 
Merrill, 1987). 

Process Evidence: 

Transactions: There would be evidence of learners hav- 
ing gained access to offered presentations on strategy or 
advice on strategic actions. 

Oral: Learners would comment on or discuss the ade- 
quacy of presently-employed strategies. 

Product Evidence : 

Journal: Entries and notations would address or docu- 
ment the adequacies or inadequacies of learning strategies 
employed. Journal comments might also allude to en- 
countered difficulties, thereby suggesting the learner's 
growing awareness of strategic insufficiency. 

Concerns: 

Learners must understand strategic applications and how 
to determine when a strategy is producing the desired 
results (Cates, 1991a; Cates, 1991b). 



Skill: Revising a learning strategy or set of 
strategies as necessary (Deny, 1985; 
Goctz, 1984; Grceno & Riley, 1987; Lawson, 
1980; Prcsseisen, 1985; Pressley et al, 1985). 

Static/Directive Support : 

Once more, the program could provide generic introduc- 
tions on the revision of strategies. It could provide direct 
instruction on revising strategies. It could use an inter- 
vening coaching function to impose revisions (Bransford 
& Vye, 1989; Marchionini, 1989; Poppen & Poppen, 
1988). 

Dynamic/Interactive Support: 

Again, the program could offer strategic advice or offer 
learners opportunities to view modeling of strategy revi- 
sion (Beyer, 1991; Derry, 1989). The program could pose 
questions designed to help focus the learner's attention on 
the key revision issues (Day et al, 1985). 



Process Evidence : 

Transactions: There would be evidence of learners hav- 
ing gained access to offered presentations on strategy or 
advice on strategic actions. 

Progress Log: The progress log would indicate purpose- 
ful backtracking to suggest that a new learning strategy 
was being applied. 

Oral: Learners would comment on or discuss changes to 
current learning strategies. 

Product Evidence : 

Journal: Entries and notations would comment on chang- 
es in strategic approach. 

Transactions: The learner's pattern of access would 
change to suggest a revision in strategy. If the learner's 
previous strategy or set of strategies had been entered and 
retained by the program, the retained strategies would 
now be different. 

Concerns : 

Learners often may become committed to strategics and 
may not realize that they can change them any time that 
they wish. Learners may believe that they must complete 
a learning episode before being able to revise their strate- 
gies. 

Cognitive Load 

Skill : Minimizing cognitive load, particular- 
ly memory load (Card et al, 1983; Cates, 
1991a; Cates, 1991c; Florin, 1990; Keller, 1987; 
Phye & Andre, 1986). 

Static/Directive Support : 

The program could maintain key information on the 
screen and on-line help would be available (Brown, 
1988). The program might make content available 
through multi-linked nodes. The program could maintain 
a record of learner position and make a "recent" map (or 
a similar device) available (Cates, 1991c). 

Dynamic/Interactive Support: 

The program could offer advice on sequence, path, or 
both. It could match its advice to the intentions and 
wishes of the learner. Advice would be "localized" or 
"compartmentalized" so that it suits exactly the context in 
which it is sought (Brown, 1988; Oren, 1990). Learners 
would not be presented with more material than they 
requested. When learners asked for advice or explanation 
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a second time, that advice or explanation would be re- 
phrased (Hutchings et al, 1992). Where possible, the 
- p.ogram would employ multi-sensory (dual) encoding, 
usually through combinations of text, graphics, and sound 
(Florin, 1990; Oren, 1990). 

Probably the best way to identify cognitive sufficiency is 
by noting the absence of cognitive overload. Cognitive 
overload might be indicated by the following types of 
evidence: 

Process Evidence: 

Transactions: There would be little evidence of recursive 
backtracking. Learners would not have viewed either 
"recent" maps or any programmed versions of internal 
structure. Students would not spend extensive periods of 
time reviewing electronic journals. 
Oral: There would be few oral requests for assistance 
and few expressions of confusion or disorientation by 
learners. 

Learner-Created: Learners would not spend long periods 
of time reviewing internal structure representations they 
have created or consulting and reviewing entries in paper- 
and-pencil journals. 

Product Evidence : 

Journal: Cognitive overload might be suggested by learn- 
ers entering notations expressing concerns about confi- 
dence or expressing confusion about the features or opera- 
tion of the product. Such evidence is a negative (contra-) 
indication of cognitive sufficiency. 
Learner -Created: Cognitively-overloaded learners might 
create memory aids. Such aids could include representa- 
tions of the internal structure of the program or "quick 
reference" cards to assist them while they're using the 
program. Once again, this type of evidence may be 
viewed as "negative evidence" suggesting that the pro- 
gram is failing to meet the learner's needs and that these 
learner actions are taken "in self-defense." 

Concerns : 

Learners need to learn memory "tricks." They may also 
need to become accustomed to the program's demands. 
Learners will need to determine the optimum combination 
of their own memory aids and the computer's external 
support mechanisms. 



Persistence and Responsibility 

Skill : Recognizing the scope of mental effort 
required and distributing mental ef- 
fort across the task as appropriate 
(Costa, 1985b; Falhikov & Boud, 1989; Hannafin 
& Rieber, 1989b; Iran-Ncjad, 1990; Milhcim & 
Martin, 1991; Presseisen, 1985; Presslcy etal, 
1985). 

Static/Directive Support : 

The program could provide information on the time de- 
mands and the relative difficulty of material to be cov- 
ered. This information might be in the form of time 
indicators (for example, clocks, stopwatches, calendar 
pages) (Cates, 1991c) or a difficulty rating score (perhaps 
from 1 to 10). The program could display progress gaug- 
es to inform learners of progress (Baecker & Small, 1990; 
Brown, 1988). 

Dynamic/Interactive Support : 

The program could provide encouraging comments as the 
learner works through the task (Costa, 1985a; Deny, 

1989) . The program's coach might offer advice or guid- 
ance on how to handle tasks. The program could offer 
presentations in which an experienced learner describes 
(models) how he or she persisted (Beyer, 1991; Florin, 

1990) . The program could employ adaptive difficulty 
levels as a way of assisting learners having troubles with 
persistence. Such adaptation might be under learner 
control (perhaps through a prompt and selection option) or 
could be automatically triggered by some symptom or set 
of symptoms that suggests flagging persistence (Marchi- 
onini, 1989; Zellermayer et al, 1991). 

Process Evidence : 

Transactions: Persistence on task would be evidenced by 
few breaks and response latencies and by times between 
screen changes which were comparable to those of learn- 
ers who work consistently and persistently. 
Oral: Discussions would be focused with little sidetrack- 
ing and few purely social interactions. 

Product Evidence : 

Journal: Entries and notations would reflect persistence 
and continuing effort. Entry dates or times would reflect 
a distribution of effort across the task. 
Progress Log: The progress log would show continuing 
progress toward the goal. 
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Transactions: If items designed to test acquired knowl- 
edge or skill were embedded in inslruction, learner perfor- 
mance on these items would be generally consistent 
across ail aspects of the task. 

Concerns : 

Lower ability learners or learners with lower levels of 
self-confidence or self-esteem may be easily discouraged. 
Computers programs are not ideally suited to providing 
the "warm" human support that such learners may need. 

Skill : Taking personal control and responsi- 
bility for learning (Falhikov & Boud, 
1989; Keller & Keller, 1991; Prawat, 1989). 

Static/Directive Support : 

The program might use phrases and expressions that 
emphasize the centrality of the learner. For example, 
instead of having the program ask the learner to indicate 
"your choice" of some options, it could have the learner 
indicate "my choice" of options. In short, language used 
in the program would be learner-referenced whenever 
possible. The program would also evidence a philosophy 
that the learner is an active participant. This would be 
most noticeable once again in phrasing, where the pro- 
gram would use active phrases for learner actions. So, 
instead of stating "You will be asked to select one of the 
following and the program will then supply you with 
related materials," the program would state "Make your 
selection to view related materials" (Keller & Suzuki, 
1988). 

Dynamic/Interact i ve Support: 

As was the case above, the program would use learner- 
referenced language and would assume an active-learner 
philosophy. The program would offer choices, not im- 
pose decisions (Yordy, 1991). The program would be 
obedient, cooperative, and non-intrusive (Brown, 1988; 
Keller, 1987). 

Process Evidence : 

Once again, learners would exhibit signs of persistence on 
task (see above). In addition, they might be expected to 
use first person pronouns and possessives in oral discus- 
sions when referring to their progress. In general, over- 
heard comments would evince a sense of control and 
responsibility. 



Product Evidence : 

Journal: Entries and notations would voice a sense of 
responsibility for completion of the task, a sense of con- - 
trol (effort related to outcome), and would not attribute 
outcomes to the behavior or control of others, nor would 
they express feelings of helplessness (Gates, 1981; Gates, 
1991b). 

Concerns : 

It is not enough for the program to use the "correct" kind 
of language if it docs not actually "practice what it 
preaches." 



Metacognitive Growth 

Skill : Analyzing the success of a learning 

outcome (Iran-Nejad, 1990; Prcsseiscn, 1985; 
Wellman, 1985). 

Static/Directive Support : 

The program could maintain records of learners* previous 
performances and these records would be available to 
them. 

Dynamic/Interactive Support : 

The program could comment on learner success (Costa, 
1985a). Learners could be debriefed by a teacher or peer 
using questions and oral discussion (Baron & Kallick, 
1985; Wilson & Cole, 1991). The computer might initiate 
or facilitate this debriefing process by printing out a "de- 
briefing log" that details strategics, progress, and other 
relevant information which might assist in the debriefing. 

Process Evidence : 

Transactions: The learner would have referred to records 
of previous performances. 
Oral: Learners would participate in discussions of 
strengths and weaknesses of their task performance and 
might discuss their perceptions of success. 

Product Evidence: 

Journal: Entries and notations would refer to evaluation 
of performance in completing the task. Journal comments 
might directly address perceptions of success or failure. 
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Concerns : 

Some learners may define success as simple completion 
or as escape from the task (Prawat, 1989). Others may 
feel they have failed even when their performance is 
acceptable because they compare themselves to ni ore- 
expert models presented by the program or by the person 
doing the debriefing (Iran-Ncjad, 1990). A common 
definition of success must be mutually agreed-upon in 
advance (Keller, 1987). 



Skill : Generalizing from one learning episode 

to others (Falhikov & Boud, 1989; Gavalck & 
Raphael, 1985; Glatthom & Baron, 1985; Tobias, 
1989; Wilson & Cole, 1991). 

Static/Directive Support : 

Once again, the program could maintain records of learn- 
ers' previous performance and make these records avail- 
able to learners (Caffarella, 1987). 

Dynamic/Interactive Support: 

The program could comment on learner success or cite 
earlier performances. Learners could work with a teacher 
or peer using questions and oral discussion to determine 
exactly what lessons have been learned from the experi- 
ence. In such sessions the results of several learning 
episodes might be reviewed and a general principle to 
account for the performance in these episodes might be 
formulated (Cates, 1991b; Costa, 1985b). The computer 
"debriefing log" mentioned, above might help in discuss- 
ing multiple episodes, as might learners' journal entries. 

Process Evidence : 

Transactions: The learner would have referred to pre- 
vious performance records. 

Oral: Learners would discuss extensions of practices and 
strategies used in the present learning episode. 

Product Evidence : 

Journal: Entries and notations would discuss possible 
extensions of the practices and strategies used in the 
present learning episode. 

Transactions: If the learner had employed strategies or 
approaches not covered in the present learning episode, 
one might infer that he or she had generalized such strate- 
gies from previous learning episodes. 



Concerns : 

We cannot know for sure if learners are generalizing from 
- one learning episode to another unless we have opportuni- 
ties to observe learners across multiple learning episodes. 

Part VI: Conclusions 

This paper has examined metacognition, ways in which 
instructional hypermedia/ multimedia programs might 
stimulate, exercise, or develop specific mctacognitivc 
skills, and the types of evidence we might expect to con- 
firm the operation of each skill. This paper has attempted 
to consolidate a broad body of literature and to synthesize 
and apply it in new ways. The work started here is clear- 
ly foundational; it is intended to assist in extending re- 
search in metacognition to hypermedia/multimedia instruc- 
tional environments. Before this paper concludes, there is 
one reservation about metacognition and learner depen- 
dency that should be noted. 

Gavalck and Raphael (1985) contended that "it is the 
transfer of control from another individual to the learner 
himself or herself that is one of Urn primary criteria sug- 
gested for determining whether metacognition is involved" 
(p. 111). In a similar vein, Wolz et al (1989) wrote, 
"While initial learning may require extensive supervision, 
once the key concepts are learned, users arc expected to 
iniliate their own goals and solicit expertise from others 
only when necessary" (p. 55), 

It is unclear whether external support structure actually 
stimulates metacognition or merely substitutes for it. 
Derry (1985), Day et al (1985), and Wellman (1985) 
expressed concern that learners might well become depen- 
dent upon the presence of the external structure and 
would not, therefore, attempt to internalize the skills. In 
fact, Yore (1986) and Whitener (1989) both concluded 
thai too much external structure could actually inhibit 
mctacognitive development by short-circuiting the process 
by which learners formulate their own strategies. Kozma 
(1987) disagreed, however, arguing that external structure 
can only help learners develop their own rnetacognitive 
skills. More research on metacognition and external 
structure is needed. If we are to meet Gavalek and Raph- 
ael's dictum that control be transferred to the learner, 
researchers must attempt to identify techniques that exer- 
cise and develop students' rnetacognitive skills to the 
point where learners are no longer dependent upon the 
presence and aid of external structure. 



Paper presented at the annual meeting of the American Educational Research Association, April 20, 1992, San Francisco, CA. 

ERIC 12 



Metacognition in Interactive Hypermedia/Multimedia Instructional Products I Page 12 



References 

Baecker, R„ & Small, I. (1990). Animation at the Interface. In B. Laurel 
(Ed.), The art of human-computer interface design- (pp. 251- 
267). Reading, MA: Addison-Wesley. 

Baird, P. (1988). HyperCard opens an electronic window on Glasgow. 
The Electronic Library, 6, 344-353. 

Baron, J., & Kallick, B. (1985). What are we looking for and how can we 
find H? In A. Costa (Ed.), Developing minds: A resource book 
for teaching thinking (pp. 279-287). Washington, D.C.: 
Association for Supervision and Curriculum Development. 

Bellanca,J. (1985). A call for staff development. In A. Costa (Ed.), 

Developing minds: A resource book for teaching thinking (pp. 
13-19). Washington, D.C.: Association for Supervision and 
Curriculum Development. 

Belland, J., Taylor, W., Canelos, J., Dwyer, F„ & Baker, P. (1985). Is 
the self-paced instructional program, via microcomputer-based 
instruction, the most effective method of addressing individual 
learning differences? Educational Communications and Tech- 
nology, 33, 185-198. 

Beyer, B. (1988). Developing a thinking skills program. Boston: Allyn & 
Bacon. 

Beyer, B. (1991). Improving student thinking while learning about the 
Civil War. In The Civil War hypermedia project design docu- 
ment (pp. 145-154). Fairfax, VA: Center for Interactive Educa- 
tional Technology, George Mason University, the Corporation 
for Public Broadcasting, and Florentine Films. 

Bogdan, R., & Biklen, S.K. (1982). Qualitative research for education: 
An introduction to theory and methods. Boston: Allyn & 
Bacon. 

Borg, W., & Gall, M. (1989). Educational research: An introduction (5th 
ed.). New York: Longman. 

Bransford, J„ & Vye, N. (1989). A perspective on cognitive research 
and its implications for instruction. In L. Resnick & L. Klopfer 
(Eds.), Toward the thinking curriculum: Current cognitive re- 
search (ASCD Yearbook). Alexandria, VA: Association for 
Supervision and Curriculum Development. 

Brown, A. (1987). Metacognition, executive control, self-regulation, and 
other more mysterious mechanisms. In K. Weinert & R. 
Kluwe (Eds.), Metacognition, motivation, and understanding 
(pp. 65-116). Hillsdale, NJ: Lawrence Eribaum. 

Brown, C ML (1988). Human-computer interface design guidelines. 
Norwood, NJ; Ablex. 

BuIfinch.T. (1968). BulHnch's mythology: The age of fable. New York: 
Doubleday. 



Caffarella, E. (1987). Evaluating the new generation of computer-related 
instructional software. Educatbnal Technology, 27(4), 19-24. 

Card, S., Moran, T., & Newell, A. (1983). The psychology of human- 
computer interaction. Hillsdale, NJ: Lawrence Eribaum. 

Carrier, C, & Jonassen, D. (1988). Adapting courseware to accommo- 
date individual differences. In D. Jonassen (Ed.), Instructional 
designs for instructional courseware (pp. 203-226). Hillsdale, 
NJ: Lawrence Eribaum. 

Case, R. (1980a). The underlying mechanism of intellectual develop- 
ment. In J. R. Kirby & J. Biggs (Eds.), Cognition, develop- 
ment, and instruction (pp. 5-37). New York: Academic Press. 

Case, R. (1980b). Implications of neo-Piagetian theory for improving the 
design of instruction. In J. R. Kirby & J. Biggs (Eds.), Cogni- 
tion, development, and instruction (pp. 161-186). New York: 
Academic Press. 

Gates, R.S. (1990, December). The Gordian knot: A parable for deci- 
sion makers. Management Review, pp. 47; 49. 

Gates, W.M. (1981). Perhaps, sorta, kinda, maybe: The student as 
faker. Tl)e Clearing House, 54, 217-218. 

Cates, W.M. (1990). Helping students learn to think critically: Detecting 
and analyzing bias in films. The Social Studies, 81, 15-18. 

Cates, W.M. (1991a, November). Fifteen principles for designing effec- 
tive instructional multimedia products. A paper presented at 
the annual meeling of the Minnesota Educational Computing 
Consortium, Minneapolis, MN. 

Cates, W.M. (1991b). What we need to teach students before they work 
on computer-assisted instruction: Lessons gleaned from CAI 
failures. International Journal of Instructional Media, 18(2), 
129-140. 

Gates, W.M. (1991c). Design and operational specifications of the 

hypermedia environment. In The Civil War hypermedia project 
design document (pp. 57-87). Fairfax, VA: Center lor Interac- 
tive Educational Technology, George Mason University, the 
Corporation for Public Broadcasting, and Florentine Films. 

Charp, S. (1986). Issues and trends in the use of computers by children 
in an information age. In B. Sendov & I. Stanchev (Eds.), 
Children in an information age: Tomorrow's problems today 
(pp. 188-194). New York: Pergamon. 

Collins, A., Brown, J.S., & Newman, S. (1989). Cognitive apprentice- 
ship: Teaching the crafts of reading, writing, and arithmetic. 
In L. Resnick (Ed.), Knowing, learning, and instruction: Es- 
says in honor of Robert Glaser. Hillsdale, NJ: Lawrence Eri- 
baum. 

Costa, A. (1985a). Thinking for, of, and about thinking. In A. Costa 

(Ed.), Developing minds: A resource book for teaching think- 
ing (pp. 20-23). Washington, D.C.: Association for Supervi- 
sion and Curriculum Development. 



Paper presented at the annual meeting of the American Educational Research Association, April 20, 1992, San Francisco, CA. 

Ej& 13 



Metacognition in Interactive Hypermedia/Multimedia Instruct onal Products I Page 13 



Costa, A. (1985b). How can we recognize improved student thinking? 
In A. Costa (Ed.), Developing winds: A resource book for 
teaching thinking (pp. 288-290). Washington, D.C.: Associa- 
tion for Supervision and -Curriculum Development. - - 

Dalton l D.,Hannafin,M., & Hooper, S. (1989). Effects of individual and 
cooperalive computer-assisted instruction on student perfor- 
mance and attitudes. Educational Technology Research and 
Development, 37(2), 15-24. 

Day, J., French, L, & Hall, L. (1985). Social influences on cognitive 
development. In D. Forrest-Pressley, G. MacKinnon, & T.G. 
Waller (Eds.), Metacognition, cognition, and human perfor- 
mance: Theoretical perspectives (Vol. 1) (pp. 33-56). Orlan- 
do, FL: Academic Press. 

DeBower, C, & DeBower, K. (1990). A decalogue for teaching mathe- 
matics (Fastback series 309). Bloomington, IN: Phi Delta 
Kappa. 

Dee, M., & BarkJey, W. (1989). Higher order thinking: Problem solving/- 
decisionmaking. Dover, DE: Delaware State Department of 
Public Instruction. 

Del Mario Rysavy, S., & Sales, G. (1991). Cooperative [earning in 
computer-based instruction. Educational Technology Re- 
search and Development, 39(2), 70-79. 

Derry, S. (1985). Strategy training: An incidental learning model for 
CAI. Journal of Instructional Development, 8(2), 16-23. 

Deny, S. (1989). Putting learning strategies to work. Educational Lead- ? 
ership, 46(4), WO. 

Entwistle, N. (1981). Styles of learning and teaching. New York: Wiley. 

Falhikov, N., & Boud, D. (1989). Student self-assessment in higher 
education: A mela-analysis. Review of Educational Re- 
search, 59, 395-430. 

Flavell. J. (1976). Metacognitive aspects of problem solving, In L. Res- 
nick (Ed.), The nature of intelligence. Hillsdale, NJ: Lawrence 
Erlbaum. 

Fleming, F,, & Levie, W. (1978). Instructional message design: Princi- 
ples from the behavioral sciences. Englewood Cliffs, NJ: 
Educational Technology, 

Flesch,R. (1981). Why Johnny still can't read: A new look at the 
scandal of our schools. New York: Harper & Row. 

Flexner, A. (1917). A modern school (Occasional paper New York: 
General Education Board. 

Florin, F. (1990). Information landscapes. In S. Ambron & K. Hooper 
(Eds.), Learning with interactive multimedia: Developing and 
using multimedia tools in education (pp. 28-49). Redmond, 
WA: Microsoft. 



Franklin, B. (1779). Proposals relating to the education of youth in 

Pennsylvania, in J. H. Best (Ed.) (198?), Benjamin Franklin 
on education (pp. 126-151). New York: Bureau of Publica- 
tions, Teachers College, Columbia University.- 

Gagne, R., & Briggs, L. (1979). Principles of instructional design (2nd 
ed.). New York: Holt, Rinehart, & Winston. 

Gavalek, J., & Raphael, T. (1985). Metacognition, instruction, and the 
role of questioning activities. In D. Forrest-Pressley, G. Mac- 
Kinnon, & T.G. Waller (Eds.), Metacognition, cognition, and 
human performance: Instructional practices (Vol. 2) (pp. 103- 
136). Orlando, FL: Academic Press. 

Giatthorn, A., & Baron, J. (1985). The good thinker. In A. Costa (Ed.), 
Developing minds: A resource book for teaching thinking (pp. 
49-53). Washington, D.C.: Association for Supervision and 
Curriculum Development. 

Glynn, S., & DiVesta, F. (1977). Outline and hierarchical organization as 
aids for study and retrieval. Journal of Educational Psycholo- 
gy, 69, 89-95. 

Goetz, E. (1984), The role of spatial strategies in processing and re- 
membering texts: A cognitive-information-processing analysis. 
In C. Holley & D. Dansereau (Eds.), Spatial learning strate- 
gies: Techniques, applications, and related issues (pp.47-77). 
Orlando, FL: Academic Press. 

Gomoll.K. (1990). Some techniques for observing users. In B. Laurel 
(Ed.), The art of human-computer interface design (pp. 85-90). 
Reading, MA: Addison-Wesley. 

Gordon, C, & Braun, C. (1985). Metacognitive processes: Reading and 
writing narrative discourse. In D. Forrest-Pressley, G. MacK- 
innon, & T.G. Waller (Eds.), Metacognition, cognition, and 
human performance: Instructional practices (Vol, 2) (pp. 1- 
75). Orlando, FL: Academic Press. 

Gray, S. (1987). The effect of sequence control on computer assisted 
learning. Journal of Computer-Based Instruction, 14(2), 54-56. 

Greeno, J., & Riley, M. (1987). Processes and development of under- 
standing. In K. Weinert & R. Kluwe (Eds.), Metacognition, 
motivation, and understanding (pp. 289-313). Hillsdale, NJ: 
Lawrence Erbaum. 

Gygi, K. (1990), Recognizing the symptoms of hypertext ... And what 
to do about it, In B. Laurel (Ed.), The art of human-computer 
interface design (pp. 279-287). Reading, MA: Addison-Wes- 
ley. 

Hannafin, M., & Rieber, L. (1989a). Psychological foundations of in- 
structional design for emerging computer-based instructional 
technologies: Part I. Educational Technology Research and Develop- 
ment, 37(2), 91-101. 

Hannafin. M., & Rieber, L. (1989b). Psychological foundations of in- 
structional design for emerging computer-based instructional 
technologies: Part II. Educational Technology Research and Devel- 
opment, 37(2), 102-114. 



Paper presented at the annual meeting of the American Educational Research Association, April 20, 1992, San Francisco, CA. 



1 



Metacognition in Interactive Hypermedia/Multimedia Instructional Products I Page 14 



Haywood, K.M. (1987). Thoughts on thinking: A critical human-re- 
source skill . Cornell Hotel and Restaurant Administration 
Quarterly, 28(2), 51-55. 

Heller, R.S. (1990). The role of hypermedia in education: A look at the 
research issues. Journal of Research on Computing in Edu- 
cation, 22, 431-441. 

Ho, C, Savenye, W., & Haas, N. (1986). The effects of orienting objec- 
tives and review on learning from interactive video. Journal of 
Computer-Based Instruction, 13, 126-129. 

Hatchings, G., Hall, W., Briggs, J., Hammond, N., Kibby, M., McKnight, 
C, & Riley, D. (1992). Authoring and evaluation of hyper- 
media for education. Computers and Education, 18, 171-177. 

Iran-Nejad, A. (1990). Active and dynamic self-regulation of learning 
processes. Review of Educational Research, 60,573-602. 

Jonassen, D. (1988). Designing structured hypertext and structuring 
access to hypermedia. Educational Technology, 28(11), 13- 
16. 



Kinzie, M., Sullivan, H., & Berdel, R. (1988). Learner control and 

achievement in science computer-assisted instruction. Com- 
puters and Education, 17(1), 13-24. 

Kozma, R. (1987). The implications of cognitive psychology for comput- 
er-based learning tools. Ann Arbor, Mi: Program on Learning, 
Teaching, and Teaching, National Center for Research to Im- 
prove Postsecondary Teaching and Learning, University of 
Michigan. 

Krahn, C, & Blanchaer, M. (1986). Using an advance organizer to 

improve knowledge application by medical students in comput- 
er-based clinical simulations. Journal of Computer-Based 
Instruction, 13(3), 71-74. 

Lanza, A., & Roselli, T. (1991). Effects of hypertextual approach versus 
the structured approach on students' achievement. Journal of 
Computer-Based Instruction, 18, 48-50. 

Lawson, M. (1980). Metamemory: Making decisions about strategies. 
In J. R. Kirby & J. Biggs (Eds.), Cognition, development, and 
instruction (pp. 145-159). New York: Academic Press. 



Jonassen, D. (1991). Hypertext as instructional design. Educational 
Technology Research and Devebpment, 39(1), 83-92. 

Jonassen, D., & Grabinger, R.S. (1990). Problems and issues in de- 
signing hypertext/hypermedia for learning. In D. Jonassen & 
H. Mandl (Eds.), Designing hypermedia for learning (pp. 5- 
25). Berlin: Springer-Verlag. 

Jones, B.F. (1992). Cognitive designs in education. In M. C. Alkin 
(Ed.), Encyclopedia of educational research (6th ed.). New 
York: Macmillan. 

Judd.C. (1933). Problems of education in the United States. New 
York: McGraw-Hill. 

Kearsley.G. (1988). Authoring considerations for hypertext. Education- 
al Technology, 28(11), 21-24. 

Keller, A. (1987). When machines teach: Designing computer course- 
ware. New York: Harper & Row. 

Keller, J., & Suzuki, K. (1988). Use of the ARCS motivation model in 
courseware design. In D. Jonassen (Ed.), Instructional de- 
signs for microcomputer courseware (pp. 401-434). Hillsdale, 
NJ: Lawrence Erlbaum. 

Keller, J., & Keller, B. (1991). Motivating learners with multimedia in- 
struction. Proceedings of the International Conference on 
Multi-Media in Education and Training, Japan Association for 
Educational Technology and International Society for Tech- 
nology in Education, Tokyo, Japan. 

Kinzie, M., & Sullivan, H. (1989). Continuing motivation, learner control, 
and GAL Educational Technology Research and Develop- 
ment, 37(2), 5-14. 



Lehrer, R..& Randle.L. (1987). Problem solving: Metacognition and 
cognition: The effects of interactive software for first-grade 
children. Journal of Educational Computing Research, 3, 409- 
427. 

Lee.MJ. (1990). Effects of different loci of instructional control on 

students' metacognition and cognition: Learner vs. program 
control. Proceedings of Selected Paper Presentations at the 
Convention of the Association for Educational Communications 
and Technology . 

Lee, M.J. (1991). Metacognitive and cognitive effects of different loci of 
instructional control. Proceedings of Selected Research 
Presentations at the Annual Convention of the Association for 
Educational Communications and Technology. 

Marchionini, G. (1988). Hypermedia and learning: Freedom and choice. 
Educational T^ology, 28(11), 8-12. 

Marchionini, G. (1989). Information seeking in electronic encyclopedias. 
Machine-Mediated Learning, 3, 211-226. 

Marzano, R. (1987). Policy constraints to the teaching of thinking. 
Journal of State Government, 60(2), 64-67. 

McNeil, B., & Nelson, K. (1991). Meta-analysis of interactive video 

instruction: A 10-year review of achievement effects. Journal 
of Computer-Based Instruction, 18, 1-6. 

Merril', J, (1987). Levels of questioning and forms of feedback: Instruc- 
tional factors in courseware design. Journal of Computer- 
Based Instruction, 14, 18-22. 

Milheim, W. ( 4 Martin, B. (1991). Theoretical bases for the use of learn- 
er control: Three different perspectives. Journal of Computer- 
Based Instruction, 18, 99-105. 



Paper presented at the annual meeting of the American Educational Research Association, April 20, 1992, San Francisco, CA. 



15 



Metacognition in Interactive Hypermedia/Multimedia Instructional Products I Page 15 



Miller, G. (1956). The magical number seven, plus or minus two: Some 
limits on our capacity for processing information. The Psycho- , 
logical Review, 63,81-97. 



Monroe, P. (1907). A brief course in the history of education. 
York: Macmillan. 



New 



Morariu.J. (1988). Hypermedia in instruction and training: The power 
and the promise. Educational Technobgy, 28(11), 17-20. 

Moss, E. (1990). Social interaction and metacognitive development in 
gifted preschoolers. Gifted Child Quarterly, 34(1), 16-20. 

Oren, T. (1990). Cognitive load in hypermedia: Designing for the ex- 
ploratory learner. In S. Ambron & K. Hooper (Eds.), Learning 
with interactive multimedia: Developing and using multimedia 
tools in education (pp. 126-136). Redmond, WA: Microsoft. 

Oren, T., Salomon, G., Kreitman, K., & Don, A. (1990). Guides: Char- 
acterizing the interlace. In B. Laurel (Ed.), The art of human- 
computer interface design (pp. 367-381). Reading, MA: Ad- 
dison-Wesley. 

O'Shea, T., & Self, J. (1983). Learning and teaching with computers. 
Englewood Cliffs, NJ; Prentice-Hall. 

Parker, W. (1991). Achieving thinking and decision-making objectives in 
social studies. In J. Shaver (Ed.), Handbook of research on 
social studies teaching and learning (Chapter 30), New York: 
Macmillan. 

Phye, G., & Andre, T. (1986). Cognition, learning, and education. In T. 
Andre and G. Phye (Eds.), Cognition and classroom learning 
(pp. 1-19). San Diego, CA: Academic Press. 

Poppen, L, & Poppen, R. (1988). The use of behavioral principles in 
educational software. Educational Technology, 28(2), 37-41. 

Prawat, R. (1989). Promoting access to knowledge, strategy, and dispo- 
sition in students: A research synthesis. Review of Educa- 
tional Research, 59, 1-41. 

Presseisen, B. (1985). Thinking skills: Meaning and models. In A. 

Costa (Ed.), Developing minds: A resource book for teaching 
thinking (pp. 43-48). Washington, D.C.: Association for 
Supervision and Curriculum Development. 

Pressley, M., Borkowski, J., & O'Sullivan, J. (1985). Children's meta- 

memory and the teaching of memory strategies. In D. Forrest- 
Pressley, G. MacKinnon, & T.G. Waller (Eds.), Metacognition, 
cognition, and human performance: Theoretical perspectives 
(Vol. 1) (pp. 11M53). Orlando, FL: Academic Press. 

Psotka.J. (1991). Hypertext and education. Educational Researcher, 
20(6), 26-27. 

Robert, M., Racine, B., & Bowers, C. (1990). A worker's mind is a 
terrible thing to waste. Quality Progress, 23(10), 59-61. 



Ross, S., & Morrison, G. (1989). In search of a happy medium in in- 
structional technology research: Issues concerning external 
validity, media replications, and learner control. Educational 
. .. Technology Research and Development, 37(1), 19-33. 

Schloss, P., Sindelar, S., Cartwright, G., & Schloss, C. (1986). Efficacy 
of higher cognitive and factual questions in computer-assisted 
instruction. Journal of Computer-Based Instruction, 13(3), 75- 
79. 

Slee, E.J. (1989, February). A review of the research on interactive 

video. A paper presented at the annual meeting of the Asso- 
ciation for Educational Communications and Technology, 
Dallas, TX. 

Stinespring, J. (1991, March). Cognition: What we Iwow and what it 
means for art teachers. A paper presented at the Convention 
of the National Art Education Association, Atlanta, GA. 

Tobias, S. (1989). Using computers to study consistency of cognitive 
processing of instruction. Computers in Human Behavior, 5, 
107-118. 

Tucker, S., & Dempsey, J. (1991). Photo-interviewing: A tool for evalu- 
ating technological innovations. Evaluation Review, 15, 639- 
654. 

Webster's new collegiate dictionary. (1976). Springfield, MA: G. & C. 
Merriam. 

Wellman, H. (1985). The origins of metacognition. In D. Forrest- Press- 
ley, G. MacKinnon, & T.G. Waller (Eds.), Metacognition, cogni- 
tion, and human performance: Theoretical perspectives (Vol. 
1) (pp. 1-31). Orlando, FL: Academic Press. 

White, M.A. (1988). The third learning revolution. Electronic Learning, 
7(4), 6-7. 

White, C. (1989, June). A field test of the hypertext product 'scientists at 
work': A report of findings. A paper presented at the annual 
National Educational Computing Conference, Boston, MA. 

White, C.,Cates,W.,&Fontana,L (1991). The Civil War interactive 
project: Meeting four major design challenges. Proceedings 
of the International Conference on Multi-Media in Education 
and Training, Japan Association for Educational Technology 
and International Society for Technology in Education, Tokyo, 
Japan. 

Whitener.E. (1989). A meta-analysis of the effect on learning of the 
interaction between prior achievement and instructional sup- 
port. Review of Educational Research, 59, 65-86. 

Wilson, B., & Cole, P. (1991, April). An instructional-design review of 
cognitive teaching models. A paper presented at the meeting 
of the American Educational Research Association, Chicago, 
IL. 



ERLC 



Paper presented at the annual meeting of the American Educational Research Association, April 20, 1992, San Francisco, CA. 

16 BEST COPY AVAILABLE 



Metacognition in Interactive Hypermedia/Multimedia Instructional Products I Page 16 



Wolz, U., McKeown, K., & Kaiser, G. (1989). Automated tutoring in 
interactive environments: A task-centered approach. Ma- 
chine-Mediated Learning, 3, 53-79. 

Yordy, L. (1991). Incorporating learning research into instructional 

program design. Journal of Educational Technology Systems, 
19, 223-231 . 

Yore, L. (1986). The effects of lesson structure and cognitive style on 
the science achievement of elementary school children. 
Science Education, 70, 461-471. 

Young, R. (1983). Surrogates and mappings: Two kinds of conceptual 
models for interactive devices. In D. Gentner & A. Stevens 
(Eds.), Mental models (pp. 35-52). Hillsdale, NJ: Lawrence 
Erlbaum. 

Zeliermayer, M. f Salomon, G., Globerson, T., & Givon, H. (1991). En- 
hancing writing-related metacognitions through a computerized 
writing partner. American Educatbnal Research Journal, 28, 
373-391. 



Paper presented at the annual meeting of the American Educational Research Association, April 20, 1992, San Francisco, CA. 



ERIC 




